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Introduction 


Refractive  errors  and  higher-order  aberrations  of  the  eye 

Until  recently,  the  clinical  measurement  and  correction  of  refractive  errors  was  limited  to  just 
spherical  and  astigmatic  errors,  while  additional  more  complicated  refractive  errors,  known  as 
higher-order  aberrations,  were  ignored.  For  most  normal  patients,  higher-order  aberrations  have 
little  effect  on  vision,  and  until  recently,  they  could  not  be  measured  or  corrected  clinically.  This 
has  changed  with  refractive  surgery,  which  was  originally  designed  to  correct  spherical  and 
astigmatic  errors.  In  many  cases,  the  surgery  inadvertently  created  large  higher-order  aberrations 
that  caused  uncorrectable  poor  vision  (Howland,  2000,  Maguire,  1994).  These  unwanted 
aberrations  were  particularly  troublesome  for  patients  with  small  treatment  zones,  higher 
prescriptions  or  large  pupils  (Boxer  Wachler,  2003,  Boxer  Wachler,  Huynh,  El-Shiaty  & 
Goldberg,  2002,  Casson,  1996,  Haw  &  Manche,  2001,  Lee,  Hu  &  Wang,  2003,  Martinez, 
Applegate,  Klyce,  McDonald,  Medina  &  Howland,  1998,  Wachler,  Durrie,  Assil  &  Krueger, 
1999).  This  also  created  concerns  for  Army  aviation  applicants.  By  regulation,  these  patients 
must  have  low  spherical  and  astigmatic  refractive  errors  but  could  have  unrecognized  large 
higher-order  aberrations.  Do  higher-order  aberrations  adversely  affect  flight  performance? 
Should  the  Army  establish  additional  visual  or  optical  standards  for  eyes  that  have  had  refractive 
surgery?  How  should  higher-order  aberrations  be  measured?  What  kind  and  magnitude  of 
higher-order  aberrations  can  be  considered  normal  and  abnormal?  Because  of  refractive  surgery, 
the  formerly  esoteric  study  of  ocular  aberrations  became  clinically  relevant  for  eye  care  and 
aviation  medicine. 


Aberrometers — instruments  that  measure  ocular  aberrations 

Higher-order  aberrations  are  more  difficult  to  measure  than  spherical  or  astigmatic  refractive 
errors.  In  fact,  until  the  recent  development  of  aberrometers — instruments  that  measure  higher- 
order  aberrations — there  was  no  clinical  method  available  for  measuring  them. 

In  the  1970s  engineers  began  to  explore  new  techniques  for  measuring  atmospheric  aberration 
affecting  telescopes.  Dr.  Roland  Shack,  of  the  University  of  Arizona,  modified  an  old  optical 
test,  the  Hartmann  test,  and  invented  what  we  now  know  as  the  Shack-Hartmann  wavefront 
sensor  (Platt  &  Shack,  2001).  Shack-Hartmann  sensors  were  fitted  on  the  worlds’  largest 
telescopes,  and  enabled  astronomers  to  significantly  improve  telescope  image  quality  (Fugate  & 
Wild,  1994). 

Dr.  Junzhong  Liang  was  the  first  person  to  use  a  Shack-Hartmann  sensor  to  measure 
aberrations  of  the  eye  (Liang,  Grimm,  Goelz  &  Bille,  1994),  and  within  a  few  years,  other  vision 
scientists  around  the  world  began  using  the  same  technique  (  Liang  &  Williams,  1997,  Liang, 
Williams  &  Miller,  1997,  Salmon  &  Thibos,  1998,  Thibos  &  Hong,  1999,  Miller,  2000,  Moreno- 
Barriuso  &  Navarro,  2000,  Hamam,  2000,  Prieto,  Vargas-Martin,  Goelz  &  Artal,  2000,  Hofer, 
Artal,  Singer,  Aragon  &  Williams,  2001,  Porter,  Guirao,  Cox  &  Williams,  2001,  Marcos,  Diaz- 
Santana,  Llorente  &  Dainty,  2002,  Thibos,  Hong,  Bradley  &  Cheng,  2002,  Yoon  &  Williams, 
2002,  Salmon,  West,  Gasser  &  Kenmore,  2003).  Although  other  technologies  have  been 
developed  to  measure  the  eye’s  higher-order  aberrations,  Shack-Hartmann-type  aberrometers  are 
the  most  popular.  In  2001,  WaveFront  Science,  Inc,  (Albuquerque,  NM)  began  marketing  the 
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Complete  Ophthalmic  Analysis  System  or  COAS  (Figure  1),  which  was  the  first  commercial 
ophthalmic  Shack-Hartmann  aberrometer. 

Clinically,  refractive  errors  are  usually  measured  by  a  nulling  technique  that  determines  the 
spherical  and  astigmatic  lenses  needed  to  negate  the  errors.  Aberrometers,  on  the  other  hand, 
measure  distortions  in  wavefronts  of  light  that  have  passed  through  the  eye’s  optics;  that  is,  they 
directly  measure  the  optical  errors,  and  they  do  so  my  measuring  the  topography  of  a  wavefront 
of  light  at  many  locations  across  the  pupil  (Salmon  &  West,  2002).  To  do  this,  Shack-Hartmann 
aberrometers  project  light  onto  the  eye,  and  then  measuring  the  light  reflected  back  out  of  the  eye 
(Figure  2a).  While  an  aberration-free  eye  will  emit  a  wavefront  with  perfectly  flat  topography, 
aberrations  in  the  eye’s  optical  system  will  bend  or  warp  the  wavefront.  Depending  on  the  kind 
of  refractive  aberrations  present,  the  wavefront  can  be  distorted  into  various,  sometimes 
complex,  shapes.  Figure  2b  illustrates  a  wavefront  that  has  been  curved  by  a  myopic  refractive 
error. 


Figure  1.  The  Complete  Ophthalmic  Analysis  System  (COAS). 
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Figure  2.  How  Shack-Hartmann  aberrometers  measure  wavefront  errors.  Shack-Hartmann 
aberrometers  projecting  a  point  onto  the  retina  and  measure  the  wavefront  of 
light  emitted  from  the  eye.  In  an  eye  with  perfect  optics,  a  flat  wavefront 
emerges  (a).  Aberrations  such  as  simple  myopia  (b)  bend  and  distort  the 
wavefront. 


Reporting  aberrations — Zernike  polynomials 

Conventional  clinical  notation  reports  refractive  errors  using  three  numbers — the  sphere, 
cylinder  and  axis  (for  astigmatism)  of  the  correcting  lens,  while  higher-order  aberrations  are 
ignored.  A  new  notation  was  needed  that  could  include  the  higher-order  refractive  errors 
(aberrations),  and  in  2000,  the  Optical  Society  of  America  (OSA)  adopted  Zernike  polynomials 
as  the  standard  system  for  specifying  aberrations  of  the  eye  (Atchison,  Scott  &  Cox,  2000, 
Thibos,  Applegate,  Schwiegerling,  Webb  &  Members,  2000).  The  OSA  standard  is  now  well 
established  internationally  among  refractive  surgeons  and  vision  scientists.  Zernike  polynomials 
break  down  the  eye’s  refractive  error  into  well-defined  sub-aberrations,  known  as  Zernike 
modes;  that  is,  each  mode  is  a  distinct  type  of  refractive  error  or  aberration.  This  system 
identifies  modes  using  either  a  single-number  index  (which  is  more  convenient  for  graphing),  or 
a  double  index  that  uses  a  subscript  and  superscript.  Depending  on  the  detail  required,  any 
number  of  Zernike  modes  may  be  included  in  the  analysis  of  an  eye’s  total  refractive  error. 

Some  papers  have  described  refractive  errors  analyzed  into  as  many  as  65  modes  (Liang  & 
Williams,  1997;  Liang  et  al.,  1997;  Salmon,  1999).  The  Zernike  modes  may  be  grouped  into  a 
hierarchy  of  orders,  which  contain  refractive  aberrations  of  increasing  complexity.  When  the 


OSA  subscript-superscript  notation  is  used,  the  subscript  indicates  the  Zernike  order,  while  the 
superscript  identifies  a  specific  aberration  within  that  order.  For  example,  Z-f 1  represents  the 
third-order  aberration,  which  is  sometimes  called  vertical  coma  (Atchison  et  al.,  2000). 


Study  objectives 

The  United  States  Army  Aeromedical  Research  Laboratory  (USAARL)  is  evaluating 
refractive  surgery  and  its  impact  on  Army  pilots.  Instrumentation  that  can  accurately  measure 
the  optics  of  the  human  eye  is  needed  to  objectively  evaluate  optical  and  visual  performance. 

The  objectives  of  this  study  were  to  evaluate  the  Complete  Ophthalmic  Analysis  System  (COAS; 
manufactured  by  Wavefront  Sciences,  Inc.,  Albuquerque,  NM)  in  terms  of  its  1)  accuracy,  2) 
repeatability  and  3)  induced  instrument  myopia  for  a  normal  population  of  Army  flight  school 
applicants  who  had  not  undergone  refractive  surgery. 

Accuracy,  sometimes  called  validity,  describes  how  correctly  an  instrument  measures  what  it 
is  supposed  to  measure.  We  evaluated  accuracy  for  spherical  defocus  and  astigmatism  only,  by 
comparing  COAS  measurements  to  data  obtained  using  standard  clinical  methods.  We  were  not 
able  to  assess  accuracy  for  higher-order  aberrations  because  we  had  no  other  way  to  measure  the 
higher-order  aberrations  than  by  using  the  COAS  aberrometer. 

Repeatability  describes  measurement  consistency.  We  assessed  repeatability  for  both  lower 
(sphere  and  astigmatism)  and  higher-order  aberrations  by  making  multiple  measurements  and 
computing  the  variance.  We  also  measured  instrument  myopia,  which  it  is  the  tendency  of  the 
eye  to  over-focus  when  looking  into  tabletop  instruments  such  as  the  COAS.  This  and  other 
automated  refractors  contain  an  internal  fixation  target  that  is  optically  projected  to  infinity,  so 
the  eye  will  focus  at  that  distance.  However,  in  spite  of  the  fact  that  the  target  is  at  infinity 
optically,  some  eyes  focus  for  a  nearer  distance  because  the  patient  knows  the  object  he’s 
viewing  is  actually  near  (inside  the  instrument).  This  contributes  to  measurement  error  because 
it  makes  the  instrument  over-estimate  myopia  or  underestimate  hyperopia. 

The  principles  of  Shack-Hartmann  aberrometry  are  well  established  and  laboratory  devices  have 
proven  reliable  (Moreno-Barriuso  &  Navarro,  2000,  Prieto  et  al.,  2000,  Salmon  &  Thibos,  1998). 
However,  apart  from  information  provided  by  the  manufacturer,  little  work  has  been  done  to  test 
accuracy  of  the  new  clinical  aberrometers,  including  the  COAS.  One  recent  study  reported  the 
accuracy,  repeatability  and  instrument  myopia  of  the  COAS  for  measurements  of  spherical 
defocus  and  astigmatism  for  a  group  of  twenty  myopic  patients  (Salmon  et  al.,  2003).  Table  1 
summarizes  the  COAS’s  accuracy  reported  in  that  study.  It  shows  measurement  error,  in 
diopters  (D)  for  pupil  diameters  of  4.0  and  6.7  mm,  as  well  as  accuracy  for  a  conventional 
autorefractor.  Table  2  shows  lower-order  repeatability  reported  in  the  same  study  for  the  COAS 
and  autorefractor,  without  and  with  cycloplegia.  COAS  Instrument  myopia  was  reported  to  be  - 
0.03  D,  better  than  that  for  the  autorefractor,  which  was  -0.21  D. 


4 


Table  1. 

Previously  reported  accuracy  for  the  COAS  for  the  sphere  and  astigmatism  (Salmon  et  al.,  2003). 


Without  cycloplegia  (D) 

With  cycloplegia  (D) 

Instrument 

Pupil 

Sphere 

Cylinder 

Vector 

Pupil 

Sphere 

Cylinder 

Vector 

COAS 

4.0 

0.00 

-0.20 

+0.33 

4.0 

+0.07 

-0.19 

+0.32 

6.7 

+0.19 

-0.23 

+0.37 

7.7 

+0.19 

-0.15 

+0.41 

Autorefractor 

3.5 

-0.02 

-0.20 

+0.34 

3.5 

+0.16 

-0.17 

+0.22 

Table  2. 

Previously  reported  repeatability  for  the  COAS  (Salmon  et  al.,  2003). 


Instrument/pupil  size 

Without  cycloplegia  (D) 

With  cycloplegia  (D) 

COAS  4-mm  pupil 

0.27 

0.16 

COAS  full  pupil 

0.24 

0.12 

Autorefractor  3.5-mm  pupil 

0.31 

0.19 

Methods 


Subjects 

We  recruited  28  volunteers  from  among  pilot  candidates  who  were  undergoing  a  Class  I 
physical  examination,  as  required  for  entry  into  flight  school.  Inclusion  criteria  were  the  same  as 
the  vision  and  ocular  health  requirements  to  enter  flight  school  in  accordance  with  AR  40-501. 
These  included 

•  Refractive  error  between  -0.75  D  of  myopia  and  +3.00  D  of  hyperopia,  and  <  0.75  D  of 
astigmatism. 

•  Uncorrected  visual  acuity  of  20/50  or  better  in  each  eye. 

•  Best  corrected  visual  acuity  of  20/20  or  better  in  each  eye. 

•  No  evidence  of  ocular  disease 

In  addition,  subjects  must  not  have  worn  rigid  contact  lenses  within  6  months,  never  have  had 
refractive  surgery,  and  have  no  medical  contraindications  to  the  use  of  cycloplegic  or  anesthetic 
eye  drops.  Our  sample  included  one  female,  and  27  male  subjects,  with  a  mean  age  ±standard 
deviation  of  24.7  ±3.3  years.  The  mean  non-cycloplegic  refractive  error  was,  +0.30  ±0.41  D 
sphere  and  -0.21  ±0.26  D  cylinder.  Cycloplegia  shifted  the  mean  spherical  refractive  to  +0.41 
±0.70  D. 


Procedures 

The  protocol  was  approved  beforehand  by  the  US  Army  Aeromedical  Research  Laboratory 
(USAARL)  Human  Use  committee.  Each  subject  provided  written  informed  consent  before 
participating.  One  of  the  tests  required  by  the  Class  I  flight  physical  is  a  measurement  of  the 
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refractive  error  with  cycloplegia.  Cycloplegic  eye  drops  are  instilled  to  temporarily  paralyze  the 
eye’s  near-focusing  mechanism,  and  thereby  ensure  more  accurate  measurements.  For  the 
purposes  of  this  study,  several  additional  tests  were  added  to  each  subject’s  flight  physical.  We 
performed  the  following  tests,  in  the  following  order,  on  each  subject’s  right  and  left  eye. 

1 )  Measurement  of  refractive  errors  without  cycloplegia 

a.  Autorefraction  using  the  Nidek  ARK-700A,  a  commonly-used  clinical  instrument 
that  objectively  estimate  the  spherical  and  astigmatic  errors  only 

b.  Conventional  clinical  subjective  measurement  of  the  spherical  and  astigmatic 
refractive  errors  by  an  optometrist 

c.  COAS  measurements  of  sphere,  astigmatism  and  higher-order  aberrations 

2)  Measurement  of  refractive  errors  with  cycloplegia 

a.  COAS  measurements  of  sphere,  astigmatism  and  higher-order  aberrations 

b.  Clinical  subjective  measurement  of  the  spherical  and  astigmatic  refractive  errors 
by  an  optometrist 

c.  Autorefraction  using  the  Nidek  ARK-700A 

For  the  COAS  measurements,  the  subject  placed  his  head  on  a  chin  rest  and  stared  at  a  fixation 
pattern  inside  the  machine,  while  the  operator  aligned  his  eye  on  a  video  monitor,  and  took  five 
measurements  within  about  one  minute.  The  COAS  was  configured  to  report  the  refractive  error 
for  a  5-mm  diameter  pupil,  and  analyze  aberrations  up  to  the  eighth  Zernike  order  (44  modes). 

All  measurements  were  made  with  dim  room  illumination.  After  each  measurement,  the  COAS 
data,  including  sphere,  cylinder,  axis,  pupil  diameter,  Zernike  coefficients  and  other  information 
were  saved  to  a  database  for  later  analysis. 

Analysis  of  accuracy 

Ideally,  when  evaluating  accuracy  of  an  instrument,  the  true  value  to  be  measured  should  be 
known.  For  example,  in  an  earlier  USAARL  study  of  videokeratoscope  accuracy,  we  compared 
the  known  dimensions  of  plastic  artificial  corneas  to  the  measurements  reported  by  the 
instrument  (Salmon,  Rash,  Mora  &  Reynolds,  2002).  An  absolute  test  of  the  aberrometer’s 
accuracy  is  not  possible  because  there  is  no  means  to  exactly  know  the  true  value  of  each  eye’s 
aberrations.  Clinical  subjective  refraction  provides  our  best  estimate  of  lower-order  aberrations 
(sphere  and  astigmatism)  but  it  does  not  measure  higher-order  aberrations.  Accuracy  testing  was 
therefore  limited  to  lower-order  aberrations  (sphere  and  cylinder).  We  compared  sphere  and 
cylinder  measured  by  the  COAS  with  the  clinical  refraction  for  each  eye.  Since  clinical 
subjective  refraction,  our  gold  standard,  is  accurate  to  only  within  ±0.25  D,  our  assessment  of 
COAS  accurate  could  not  be  better  than  this.  COAS  accuracy  was  expressed  as  the  presumed 
error,  that  is,  the  difference  between  the  COAS  and  clinically  determined  values  for  the  sphere 
and  astigmatism. 

We  processed  the  refraction  data  (sphere,  cylinder  and  axis  for  each  eye)  according  to  the 
following  steps  (Salmon  et  al.,  2003). 

1)  Convert  sphere,  cylinder  and  axis  to  power  vectors. 
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The  standard  clinical  notation  for  recording  refractive  errors  (sphere,  cylinder  and  axis)  is 
not  well  suited  to  statistical  analysis,  since  it  does  not  allow  the  direct  computation  of 
differences,  means  or  variances.  Therefore  we  transformed  sphere  (S),  cylinder  (C)  and 
axis  (0)  into  three  quantities,  referred  to  as  J45,  M  and  Jiso,  according  to  Eqs.  (1-3).  These 
constitute  the  components  of  a  vector  that  allows  statistical  analysis  (Thibos,  Wheeler  & 
Horner,  1997).  All  subjective  refraction  data  were  treated  in  this  way.  The  subjective 
refraction  for  each  eye  was  converted  to  a  vector  quantity  labeled  vector  S. 


J45  =  (-C/2)sin(20) 

(1) 

M  =  S  +  C/2 

(2) 

J 1  so  =  (-C/2)cos(20) 

(3) 

2)  Five  COAS  measurements  were  made  of  each  eye,  and  these  measurements  were 
likewise  converted  to  dioptric  power  vectors  (by  Eqs  (1-3),  above).  We  computed  the 
mean  J45,  M  and  J|8o,  of  the  five  measurements  (mean  COAS  power  vector)  for  the  mean 
COAS  refraction  for  each  eye.  This  vector  quantity  was  labeled  vector  C. 

3)  COAS  error  was  defined  as  the  difference  (vector  E)  between  the  COAS  (vector  C)  and 
subjective  refractions  (vector  S),  according  to  Equation  4. 

E  =  C-S  (4) 

4)  Finally  we  computed  the  overall  mean  COAS  error  by  averaging  vector  E  across  all  eyes. 
To  simplify  interpretation,  we  converted  this  vector  quantity  back  to  the  clinical  notation 
using  sphere,  cylinder  and  axis  by  Eqs.  (5-7). 


c=-2V4+40 

(5) 

S  =  M-C/2 

(6) 

0  =  [tan  1  (J  45/  J  i  8o)]/2 

(7) 

To  ensure  that  the  axis  computed  by  Equation  7  conformed  to  standard  clinical  minus- 
cylinder  notation  (0>9>180  degrees),  we  corrected  the  axis  value  based  on  the  initial 
result  for  0  (Equation  7)  and  the  logical  tests  below. 

IF  J 1  so  <  0,  axis  =  0+90 
IF  Jigo  =  0  AND  IF  J45  <  0,  axis  =  135 
IF  Jiso  =  0  AND  IF  J45  >  0,  axis  =  45 
IF  J 1  so >  0  AND  IF  J45<0,  axis  =  0+180 
IF  Jigo  >  0  AND  IF  J45  >  0,  axis  =  0 
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5)  We  also  computed  the  magnitude  (m)  of  the  COAS  error  vector  (vector  E,  above)  for 
each  eye,  according  to  Equation  8,  where  J45,  M  and  J 1  so  were  components  of  vector  E. 
This  simplified  interpretation  by  converting  the  three-number  vector  quantity  E  to  a 
single-number  metric,  magnitude  m.  We  then  computed  the  mean  magnitude  of  these 
errors  across  eyes. 


m  -yj  J45  +i/|g0  (8) 

The  default/standard  COAS  computation  of  spherical  refractive  power  is  based  on  the  value  of 
mode  Zi0  (defocus)  only,  but  the  COAS  offers  an  alternate  computation  referred  to  as  the  “Seidel 
sphere.”  This  includes  mode  Z4°  (spherical  aberration)  in  the  calculation,  and  may  better  match 
the  way  a  human  eye  responds  to  a  subjective  refraction.  A  complete  description  of  how  the 
COAS  computes  both  the  standard  and  Seidel  spheres  may  be  found  in  another  article  (Salmon  et 
al.,  2003).  We  tested  COAS  accuracy  for  both  the  standard  and  the  Seidel  sphere.  For 
comparison,  we  also  tested  accuracy  of  the  autorefractor  by  following  same  procedures  for  three 
autorefractor  measurements  of  each  eye.  Accuracy  was  analyzed  separately  for  right  and  left 
eyes  with  and  without  cycloplegia. 


Repeatability  for  lower-order  aberrations 

We  evaluated  COAS  repeatability  for  the  lower-order  aberrations  (sphere  and  astigmatism) 
and  higher-order  aberration  separately.  For  each  eye  (right  and  left)  and  condition  (without  and 
with  cycloplegia),  five  measurements  with  the  COAS  were  taken  within  about  one  minute.  We 
analyzed  repeatability  for  sphere  and  astigmatism  using  both  the  standard  and  Seidel  sphere 
powers.  Autorefractor  repeatability  was  computed  in  the  same  manner,  except  that  only  three 
autorefractor  measurements  were  taken  of  each  eye.  We  processed  lower-order  data  for  each  eye 
as  follows. 

1)  Convert  each  of  the  five  COAS  refractions  (sphere,  cylinder,  axis)  to  a  power  vector 
(Equations  1-3). 

2)  Compute  the  mean  COAS  refraction  as  the  mean  of  the  five  original  power  vectors. 

3)  Subtract  the  mean  from  each  of  the  five  original  power  vectors.  This  gave  five  difference 
vectors. 

4)  Compute  the  magnitude  (Equation  8)  of  each  difference  vector  and  the  mean  of  these  five 
magnitudes.  This  gave  the  mean  deviation,  in  diopters  (D),  for  each  eye. 

5)  Square  and  sum  the  mean  deviations  for  28  eyes  and  divide  by  28  to  obtain  the  RMS 
(root  mean  squared)  deviation. 

6)  Compute  a  repeatability  coefficient,  defined  as  the  RMS  deviation  multiplied  by  1.96. 

This  analysis  generally  follows  the  methods  described  by  Bland  and  Altman  (1986)  and 
used  in  other  clinical  studies  to  evaluate  repeatability  of  diagnostic  instruments 
(Rosenfield  &  Chiu,  1995,  Walline,  Kinney,  Zadnik  &  Mutti,  1999,  Zadnik,  Mutti  & 
Adams,  1992). 
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Repeatability  for  higher-order  aberrations 


Higher-order  aberrations  (Zemike  Orders  3-8  containing  Modes  6-44  or  Z3'3  through  Z88) 
have  no  clinically  familiar  nomenclature,  so  we  performed  repeatability  statistics  directly  on 
their  Zernike  coefficients.  Five  CO  AS  measurements  were  made  of  each  eye/condition  and  this 

*  yielded  five  strings  of  39  Zernike  coefficients.  We  computed  a  standard  deviation,  standard  error 
and  finally  a  95%  confidence  interval  for  each  mode,  and  then  averaged  the  confidence  intervals 
across  28  eyes.  The  mean  95%  confidence  intervals  for  each  mode  were  interpreted  as  a 

*  measure  of  instrument  noise  and  repeatability  for  higher-order  aberration.  This  process  was 
applied  separately  to  right  and  left  eyes,  without  and  with  cycloplegia. 

Instrument  myopia 

We  computed  instrument  myopia  for  the  COAS  and  the  autorefractor  in  the  following  steps. 

1)  Cycloplegia  could  have  induced  a  slight  change  in  the  true  refractive  error,  so  for  each 
eye,  we  computed  any  such  change  (vector  A)  as  the  difference  between  the  subjective 
non-cycloplegic  (vector  Sm)  and  subjective  cycloplegic  (vector  Se)  power  vectors 
Equation  13. 

A  =  Sm-Sc  (13) 

2)  Instrument  myopia,  vector  I  in  Equation  14,  for  each  eye  was  defined  as  the  difference 
between  the  COAS  non-cycloplegic  (vector  Cm)  and  COAS  cycloplegic  (vector  Cc) 
refractions  minus  the  true  change,  vector  A  from  Equation  13. 

l=C„,-Cc-A  (14) 


The  mean  of  all  COAS  instrument  myopia  values  was  computed  for  right  and  left  eyes  when  the 
standard  and  Seidel  sphere  values  were  used.  Mean  instrument  myopia  power  vectors  were 
converted  to  sphere,  cylinder  and  axis  and  the  mean  spherical  equivalent  power. 


Results 


Accuracy 

Table  3  summarizes  the  statistics  for  COAS  accuracy  for  spherical  and  astigmatic  (cylinder) 
refractive  error.  For  comparison.  Table  4  summarizes  accuracy  for  the  autorefractor. 

The  magnitude  of  the  mean  error  vector  (Table  3,  right  column)  provides  a  single-number  metric 
to  judge  COAS  overall  accuracy  for  measuring  spherical  plus  astigmatic  refractive  error.  For  all 
eyes  and  conditions  the  error  was  less  than  0.5  D,  which  is  similar  to  the  range  of  error  seen  with 
the  autorefractor  (Table  4).  Smallest  COAS  error  was  found  when  cycloplegia  was  used, 
without  the  Seidel  option  (vector  error  about  0.3  D).  Otherwise,  without  cycloplegia,  power 
vector  error  using  either  the  standard  or  Seidel  power  was  about  0.4  D. 
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Table  3. 

COAS  lower-order  accuracy. 


Eye/condition 

Sphere  method 

Mean  sph  error 

Mean  cyl  error 

Mag  vector  error 

OD/no  cyclo 

Standard 

-0.10  ±0.60 

-0.09  ±0.27 

0.43  ±0.25 

OS/no  cyclo 

Standard 

-0.14  ±0.64 

-0.07  ±0.34 

0.43  ±0.30 

OD/no  cyclo 

Seidel 

+0.08  ±0.55 

-0.09  ±0.27 

0.38  ±0.22 

OS/no  cyclo 

Seidel 

+0.08  ±0.64 

-0.07  ±0.34 

0.43  ±0.24 

OD/cyclo 

Standard 

+0.1 4  ±0.42 

-0.08  ±0.31 

0.29  ±0.14 

OS/cyclo 

Standard 

+0.11  ±0.46 

-0.09  ±0.32 

0.29  ±0.20 

OD/cyclo 

Seidel 

+0.44  ±0.42 

-0.08  ±0.30 

0.45  ±0.24 

OS/cyclo 

Seidel 

+0.41  ±0.40 

-0.09  ±0.32 

0.41  ±0.22 

Note:  The  table  shows  accuracy  for  measuring  sphere  and  astigmatic  refractive  error  for 
right  (OD)  and  left  (OS)  eyes,  without  and  with  cycloplegia,  using  the  standard  or  Seidel 
option  for  spherical  power.  Values  are  in  diopters  (D)  plus  or  minus  one  standard 
deviation.  A  negative  value  for  the  mean  spherical  error  shows  that  the  instrument 
overestimated  myopia.  A  positive  error  indicates  that  it  overestimated  hyperopia.  The 
last  column  shows  the  magnitude  of  the  error  when  expressed  as  a  power  vector.  COAS 
analysis  pupil  diameter  was  5.0  mm. 


Table  4, 

Autorefractor  accuracy. 


Eye/condition 

Sphere  error  (D) 

Cylinder  error  (D) 

Vector  magnitude  (D) 

OD/no  cyclo 

+0.29 

-0.15 

0.40 

OS/no  cyclo 

+0.28 

-0.12 

0.41 

OD/cyclo 

+0.46 

-0.09 

0.47 

OS/cyclo 

+0.43 

-0.05 

0.47 

Note:  A  negative  sphere  error  indicates  the  instrument  overestimated  myopia; 
a  positive  error  indicates  hyperopic  error.  The  last  column  shows  the  mean 
error  in  terms  of  power  vector  magnitude  (bold).  The  autorefractor  measured 
across  a  3.5-mm  diameter  pupil. 


For  all  eyes  and  conditions,  mean  cylinder  error  was  less  than  -0.1  D.  Without  cycloplegia, 
the  COAS  standard  sphere  tended  to  overestimate  myopia  by  about  -0.1  D.  Cycloplegia  shifted 
the  mean  error  for  standard  sphere  to  about  +0. 1 .  In  general  the  Seidel  option  shifted  the  mean 
spherical  error  about  +0.25  D  for  a  5.0-mm  diameter  pupil.  Poorest  accuracy  for  the  sphere 
power  was  seen  when  the  Seidel  option  was  used  with  cycloplegia. 

Figure  3  shows  the  distribution  of  COAS  vector  errors  across  a  range  of  spherical  refractive 
errors  (right  and  left,  standard  and  Seidel  spheres)  when  no  cycloplegia  was  used.  For  both  the 
standard  sphere  (black  symbols)  and  Seidel  sphere  (white  symbols),  approximately  80%  of  the 
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errors  were  less  than  0.60  D.  In  a  few  cases  the  vector  error  exceeded  1.00  D.  The  Seidel 
sphere  resulted  in  fewer  extreme  errors,  so  that  none  exceeded  0.90  D. 
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Figure  3.  Distribution  of  CO  AS  refractive  vector  errors  (without  cycloplegia).  Approximately 
80%  of  the  errors,  using  both  the  standard  (large  symbols)  and  Seidel  spheres 
(small  symbols)  are  within  0.60  D. 


Figure  4  shows  a  similar  plot  when  cycloplegia  was  used.  The  distribution  of  errors  was 
concentrated  closer  to  zero,  with  approximately  90%  of  the  errors  less  than  0.60  D.  In  this  case, 
fewer  outliers  were  seen  when  the  standard  sphere,  rather  than  Seidel  sphere  was  used.  In  both 
Figures  3  and  4,  the  larger  errors  are  on  the  right  side  of  the  distributions,  that  is,  for  eyes  with 
very  low  myopia  or  hyperopia. 
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Subjective  spherical  equivalent  (D) 
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Figure  4.  Distribution  of  COAS  refractive  vector  errors  (with  cycloplegia).  Approximately 
90%  of  the  errors,  using  both  the  standard  sphere  (large  symbols)  and  Seidel  sphere 
(small  symbols)  are  within  0.60  D. 


For  comparison,  Figure  5  shows  a  similar  analysis  for  the  autorefractor  errors.  The 
distribution  of  errors  is  generally  similar  to  those  of  the  COAS,  perhaps  marginally  better  than 
the  COAS  without  cycloplegia  and  marginally  worse  than  the  COAS  with  cycloplegia. 
Cycloplegia  did  not  significantly  improve  the  error  distribution  for  the  autorefractor.  About  80% 
of  the  errors  were  less  than  0.60  D.  The  autorefractor  measured  across  a  3.5-mm  diameter  pupil. 
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Error  vector  magnitude  (D) 


Figure  5.  Distribution  of  autorefractor  refractive  vector  errors.  Errors  for  right  and  left  eyes 
are  shown  when  measured  without  (circles)  and  with  (diamonds)  cycloplegia. 
Without  cycloplegia,  84%  of  the  errors  were  less  than  0.60  D.  With  cycloplegia, 
about  80%  of  the  errors  were  less  than  0.60  D. 
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Repeatability 


Table  5  shows  repeatability  of  the  COAS  for  five  measurements  taken  within  about  one 
minute.  Repeatability  is  quantified  using  the  coefficient  of  repeatability,  which  was  described  in 
the  methods  section.  For  comparison,  repeatability  coefficients  for  the  autorefractor  are  also 
listed  in  Table  5  (three  measurements).  The  autorefractor’s  repeatability  coefficients  were  about 
0.1  D  smaller  than  those  of  the  COAS. 


Table  5. 

COAS  lower-order  repeatability. 


Eye/condition 

COAS  standard  sphere  (D) 

COAS  Seidel  sphere  (D) 

Autorefractor  (D) 

OD/  no  cyclo 

0.20 

0.24 

0.12 

OS/  no  cyclo 

0.18 

0.23 

0.20 

OD/  cyclo 

0.12 

0.18 

0.08 

OS/  cyclo 

0.18 

0.22 

0.09 

Note:  Values  are  shown  for  the  standard  anc 

Seidel  options  for  computing  spherical 

power,  as  well  as  for  autorefraction.  Analysis  pupil  diameter  was  5.0  mm. 


Figure  6  shows  COAS  repeatability  for  higher-order  aberrations.  Data  points  indicate 
repeatability  in  terms  of  the  mean  95%  confidence  intervals  for  each  eye/condition  for  each 
mode.  Approximately  90%  of  the  points  fall  within  the  shaded  region,  which  generalizes  mode- 
by-mode  repeatability  based  on  these  results.  That  is,  COAS  repeatability  for  each  mode  is 
indicated  by  the  height  of  the  shaded  regions  (in  pm),  which  declines  in  each  successive  order. 
The  respective  values  for  the  third,  fourth,  fifth,  sixth  and  sevenths  orders  are:  0.035,  0.025, 
0.02,  0.015  and  0.010  pm.  Eighth-order  values  (not  shown)  were  similar  to  those  in  the  seventh 
order.  These  values  can  be  interpreted  as  our  estimate  of  measurement  noise;  that  is,  variability 
caused  by  the  instrument  or  measurement  procedure. 


Instrument  myopia 

Instrument  myopia  had  little  effect  on  astigmatism  (less  than  0. 1  D  change  in  each  case),  so 
we  summarized  instrument  myopia  in  terms  of  the  spherical  equivalent  power  (Table  6).  COAS 
instrument  myopia  was  smaller  when  the  default  sphere  was  used,  about  -0.25  D.  For 
comparison  instrument  myopia  with  the  autorefractor  was  about  -0.2  D. 
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Figure  6.  COAS  higher-order  repeatability.  The  shaded  region  contains  90%  of  the  data 

points  and  indicates  a  generalized  estimate  instrument  noise  for  each  mode.  Pupil 
diameter  was  5.0  mm. 


Discussion 


Aberrometers,  such  as  the  COAS,  provide  the  only  means  for  measuring  higher-order 
aberrations  in  a  clinical  setting,  and  the  COAS  was  chosen  by  USAARL  to  study  the  ocular 
aberrations  of  Army  pilots.  Before  scientists  or  doctors  can  depend  on  the  data  provided  by 
aberrometers,  they  need  to  know  how  reliable  these  instruments  are.  Since  aberrometers 
measure  lower-order  aberrations  as  well,  they  can  also  function  as  autorefractors  that  estimate  a 
patient’s  spectacle  prescription.  We  assessed  COAS  accuracy,  repeatability  and  instrument 
myopia  for  lower-order  aberrations  and  only  repeatability  for  higher-order  aberrations.  All 
COAS  analysis  was  done  for  a  5.0-mm  diameter  pupil. 
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Table  6. 

Instrument  myopia  for  the  COAS  and  autorefractor. 


Instrument 

Eye 

Sphere  method 

Inst  myopia  (D) 

COAS 

OD 

standard 

-0.24  ±0.42 

COAS 

OS 

standard 

-0.24  ±0.35 

COAS 

OD 

Seidel 

-0.36  ±0.43 

COAS 

OS 

Seidel 

-0.29  ±0.41 

Autorefractor 

OD 

NA 

-0.19  ±0.33 

Autorefractor 

OS 

NA 

-0.19  ±0.37 

Note:  Values  show  instrument  myopia  in  terms  of  spherical  equivalent  power 
with  standard  deviations.  A  negative  value  indicates  that,  on  average,  the 
instrument  showed  more  myopia  (less  hyperopia),  than  the  true  distance 
refractive  error. 


Accuracy 

We  expressed  accuracy  for  measuring  the  combined  lower-order  aberrations  of  sphere  and 
astigmatism  by  the  magnitude  of  the  mean  error  vector.  COAS  accuracy  was  best  with  the 
default  sphere  and  cycloplegia — 0.3  D  by  this  statistic  (Table  3).  The  autorefractor’s  best 
accuracy  was  0.4  D  (Table  4),  but  this  was  found  without  cycloplegia.  To  put  this  into 
perspective,  a  0.3-D  error  vector  magnitude  is  equivalent  to  a  1/8-D  error  in  both  the  sphere  and 
cylinder  combined  with  a  12-degree  axis  error.  This  would  be  considered  a  very  small  error 
clinically.  Most  spectacle  prescriptions  are  written  in  0.25  D  power  increments,  because  it  is 
difficult  to  subjectively  measure  spherical  or  astigmatic  refractive  error  with  an  accuracy  of  less 
than  0.25  D.  Thus,  we  found  that,  on  average,  the  COAS  was  capable  of  the  same  level  off 
accuracy  we  can  expect  for  a  conventional  clinical  subjective  refraction.  In  some  cases, 
however,  COAS  and  autorefractor  error  vector  magnitudes  of  about  1 .0  D  (Figures  2-4),  which  is 
equivalent  to  a  0.5-D  error  in  both  the  sphere  and  cylinder  with  a  30-degree  axis  error.  Our 
results  with  human  eyes  were  only  marginally  worse  than  the  COAS’  reported  accuracy  with 
model  eyes.  Cheng  (  2003)  reported  mean  errors  of  ±0.1  D  sphere,  ±0.1D  cylinder  and  ±2°  axis 
(equivalent  to  a  0.16-D  vector  error)  across  a  broad  range  of  refractive  errors  (-4.00  to  +3.00  D) 
on  model  eyes.  Accuracy  declined  slightly  for  greater  refractive  errors  in  that  study. 

There  is  still  debate  among  vision  scientists  about  how  to  best  estimate  the  clinical  subjective 
sphere  from  aberrometer  data.  The  COAS  standard  setting  computes  the  sphere  directly  from  the 
second-order  aberration  Z2°  (defocus),  while  the  Seidel-sphere  option  takes  into  account  the 
fourth-order  aberration  Z4°  (spherical  aberration).  Some  scientists  believe  that  the  Seidel  sphere 
should  give  a  better  estimate  of  the  subjective  sphere,  especially  with  large  pupils.  When  no 
cycloplegia  was  used,  we  did  not  find  better  accuracy  with  the  Seidel  sphere.  With  cycloplegia  it 
was  marginally  worse  than  with  the  default  sphere.  It’s  possible  that  our  pupil  diameters  (5.0 
mm)  were  not  large  enough  to  benefit  from  the  Seidel  computation,  since,  in  another  study,  we 
found  slightly  better  accuracy  in  larger  pupils  with  the  Seidel  sphere  (Salmon  et  al.,  2003).  It 
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appears  therefore  that  users  should  normally  leave  the  standard  sphere  setting  in  place  but 
consider  using  the  Seidel  option  for  widely  dilated  (>6  mm)  pupils. 

As  mentioned  in  the  Methods  section,  we  were  not  able  to  evaluate  accuracy  for  higher-order 
aberrations  in  this  study.  Cheng  (2003)  measured  COAS  accuracy  for  some  higher-order  modes 
r  using  model  eyes,  for  which  the  higher-order  aberrations  could  be  computed  by  ray-tracing  She 

found  mean  errors  of  <0.01  pm  for  Z40  (spherical  aberration),  <0.03  for  Z31  (coma)  and  <  0.3 
pm  for  Z42  (5.0-mm  pupil).  These  correspond  to  the  respective  equivalent  diopter  values  of  0.1, 
0.3  and  0.3  D  (Thibos  et  al.,  2002). 


Repeatability 

Repeatability  refers  to  the  variability  of  repeated  measurements.  The  COAS  repeatability 
coefficients  for  lower-order  aberrations  (<  0.25  D)  were  similar  to  what  we  would  expect  from  a 
normal  clinical  subjective  refraction.  Since  COAS  repeatability  was  marginally  better  with  the 
default  sphere  (mean  0.17  D),  we  recommend  using  it  rather  than  the  Seidel  sphere  when  trying 
to  optimize  repeatability. 

We  also  evaluated  COAS  repeatability  for  higher-order  aberrations.  Higher-order 
repeatability  was  generally  the  same  with  or  without  cycloplegia.  As  shown  by  the  shaded 
region  in  Figure  5,  it  was  approximately  0.035  pm  (equivalent  to  0.04  D)  for  third-order  modes, 
0.025  pm  (0.03  D)  for  fourth-order  modes  and  declined  to  <0.02  pm  (0.02  D)  for  the  fifth  order 
and  above.  These  values  are  important  to  keep  in  mind  when  interpreting  aberrometry,  because 
Zernike  coefficients  less  than  the  noise  level  are  essentially  unmeasurable.  For  example,  if  the 
instrument  reports  -0.02  pm  of  mode  Z3'1  (vertical  coma)  aberrations,  one  may  not  assume  that, 
the  patient  has  any  of  this  aberration,  since  it  could  just  be  due  to  instrument  noise.  These  results 
apply  for  a  pupil  diameter  of  5.0  mm.  Noise  would  increase  with  larger  pupils  and  decrease  with 
smaller  pupils. 

The  variability  that  we  measured  for  higher-order  aberrations  was  only  slightly  worse  that 
that  reported  by  Cheng  et  al.  (2004)  They  estimated  that  most  of  the  COAS’  variability  was 
attributable  to  fluctuations  in  accommodation,  the  tear  film  or  eye  position  rather  than  due  to  the 
instrument  itself.  The  COAS  may  be  subject  to  axial,  transverse  or  angular  positioning  errors, 
but  they  demonstrated  that,  within  the  range  of  misalignments  expected  for  normal  clinical  use, 
these  caused  no  significant  error. 


Instrument  myopia 

The  COAS  induced  marginally  more  instrument  myopia  than  the  autorefractor  (mean  0.28 
versus  0. 19  D).  We  previously  found  almost  no  instrument  myopia  with  the  COAS,  but  all  of 
those  subjects  were  myopic  (Salmon  et  al.,  2003).  The  slightly  greater  instrument  myopia  in 
this  study  may  have  been  caused  by  the  inclusion  of  young  hyperopes,  who  tend  to  over¬ 
accommodate  habitually. 
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Conclusions 


In  normal  eyes  higher-order  aberration  are  very  small  and  have  little  effect  on  vision.  The 
magnitudes  of  higher-order  aberrations  in  normal  eyes  (averaged  across  several  studies),  for  5.0- 
and  6.0-mm  pupils,  are  shown  in  Figure  7  (Salmon  &  van  de  Pol,  2004).  The  figure  also  shows 
the  noise  range  we  determined  for  a  5.0-mm  pupil.  For  a  5.0-mm  pupil,  the  most  prominent 
aberrations — all  third-order  modes  and  fourth-order  mode  Z40  (spherical  aberration) — are 
measurable  by  the  COAS  since  they  exceed  measurement  noise  (shaded  zone).  However,  for  a 
5.0-mm  pupil,  the  other  fourth-order  and  fifth-order  aberrations  are  so  small  that  they  don’t 
exceed  the  noise  limits,  and  would  therefore  be  difficult  to  measure.  In  order  to  detect  the  subtle 
aberrations  in  an  eye  with  good  optics,  we  therefore  recommend  measuring  with  as  large  a  pupil 
as  possible.  Clinicians,  however,  are  primarily  interested  in  measuring  abnormal  aberrations, 
which  would  be  larger  than  the  mean  values  plotted  in  Figure  7.  Based  on  this  analysis,  all 
problematic  aberrations  should  be  easily  measurable  with  the  COAS.  This  is  particularly 
relevant  for  refractive  surgery  or  in  other  cases  of  subnormal  vision  caused  by  poor  optics. 


Figure  7.  Magnitude  of  normal  higher-order  aberrations.  Mean  values  expected  for  normal 
eyes  with  5.0  and  6.0-mm  diameter  pupils  (Salmon  &  van  de  Pol,  2004). 

In  addition  to  its  capacity  to  measure  higher-order  aberrations,  the  COAS  can  serve  as  an 
autorefractor  by  measuring  sphere  and  astigmatism.  Like  a  conventional  autorefractor,  it 
occasionally  has  larger-than-average  measurement  errors,  so  when  accuracy  is  critical,  we 
recommend  comparing  COAS  refractions  to  that  obtained  by  a  careful  subjective  refraction.  We 
recommend  using  the  standard  rather  than  the  Seidel  sphere  setting,  except  for  very  large  pupils. 
Cycloplegia  slightly  improves  accuracy  and  repeatability  for  measuring  sphere  and  astigmatism, 
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but  it  is  important  to  note  that  cycloplegia  itself  can  change  the  aberrations  slightly  (Carkeet, 
2003).  Finally,  we  conclude  that  the  CO  AS  aberrometer  provides  scientists  and  clinicians  with 
an  accurate  and  repeatable  way  to  objectively  measure  lower-  and  higher-order  aberrations  of 
eyes. 
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